PLEASE DO NOT CITE OR QUOTE WIHOUT PERMISSION Chapter 3: Rules for Identifying the Cream while Producing the Sacred Or Pragmatic Fairness: Producing the Sacred by Playing by the Rules

It’s just very pragmatic. You put twelve pretty smart people in a windowless room in [some city] for two days and you expect them to remain sane. They’re pretty professional people, so they do their best. They have very different tastes so there’s a lot of potential for conflict. They stay cool-headed and they have to make these heroic efforts to agree across big disciplinary differences on what constitutes a good proposal . . . We spend so much time on it and that there’s so many people focusing at once on this proposal. . . . This year particularly, there was a lot of movement: You had a lot of situations where something was ranked relatively high [prior to the meeting and], weaknesses were discussed and people in favor were convinced [not to fund]. And conversely, something [ranked] quite low, the more you talked about it, the more it looked like you should give it a chance, that it had some promise . . . It would be sort of self-congratulatory to say that cream rises to the top and that we picked exactly the right set. I don’t think that happened. But I think we chose on average the better proposals, and maybe they’re one or two here or there that maybe someone could say, “Well, you really shouldn’t have missed this,” or “You shouldn’t have funded that.” – Economist